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Abstract 

Background: National malaria control programmes and their partners must document progress associated with 
investments in malaria control. While documentation has been achieved through population-based surveys for 
most interventions, measuring changes in malaria case management has been challenging because the increasing 
use of diagnostic tests reduces the denominator of febrile children who should receive anti-malarial treatment. 
Thus the widely used indicator, "proportion of children under five with fever in the last two weeks who received 
anti-malarial treatment according to national policy within 24 hours from onset of fever" is no longer relevant. 

Methods: An alternative sequence of indicators using a systems effectiveness approach was examined using data 
from nationally representative surveys in Zambia: the 2012 population-based Malaria Indictor Survey (MIS) and the 
201 1 Health Facility Survey (HFS). The MIS measured fever treatment-seeking behaviour among 972 children under 
five years (CU5) and 1,848 people age five years and above. The HFS assessed management of 435 CU5 and 429 
people age five and above with fever/history of fever seeking care at 149 health facilities. Consultation observation 
and exit interviews measured use of diagnostic tests, artemisinin combination therapy (ACT) prescription, and 
patient comprehension of prescribed regimens. 

Results: Systems effectiveness for malaria case management among CU5 was estimated as follows: [100% ACT 
efficacy] x [55% fever treatment-seeking from an appropriate provider (MIS)] x [71% malaria blood testing (HFS)] x 
[86% ACT prescription for positive cases (HFS)] x [73% patient comprehension of prescribed ACT drug regimens 
(HFS)] = 25%. Systems effectiveness for malaria case management among people age five and above was estimated 
at 1 5%. 

Conclusions: Tracking progress in malaria case management coverage can no longer rely solely on population- 
based surveys; the way forward likely entails household surveys to track trends in fever treatment-seeking 
behaviour, and facility/provider data to track appropriate management of febrile patients. Applying health facility 
and population-based data to the systems effectiveness framework provides a cogent and feasible approach to 
documenting malaria case management coverage and identifying gaps to direct program action. In Zambia, this 
approach identified treatment-seeking behaviour as the largest contributor to reduction in systems effectiveness for 
malaria case management. 
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Background 

Malaria has been a pervasive scourge across the cen- 
turies, particularly in Africa, however tremendous ad- 
vances in malaria control have been achieved in recent 
years [1,2]. The improvements have been most notably 
documented in the coverage of malaria prevention in- 
terventions and have led to marked declines in malaria- 
related child mortality. Despite this progress, malaria 
remains a leading cause of death among children under 
five [3], and a leading contributor to the global burden of 
disease, particularly in sub-Saharan African [4]. The esti- 
mated number of malaria cases in Africa in 2010 was 174 
million [1]. Families, communities and health systems face 
an ongoing burden of managing malaria infections. The 
recently introduced highly efficacious artemisinin combin- 
ation therapy (ACT) is now the first-line treatment in 
most countries where Plasmodium falciparum is endemic 
[1]. More recently, quality controlled rapid diagnostic tests 
(RDTs) have proven valuable in confirming infections and 
directing appropriate malaria treatment such that, along 
with microscopy, diagnostic confirmation of malaria is 
recommended worldwide [5]. 

Improvements in malaria control have been documented 
using population-based surveys to chart changes in inter- 
vention coverage [2,6]. These include the Demographic 
and Health Survey (DHS), the Malaria Indicator Survey 
(MIS), and the UNICEF Multiple Indicator Cluster Survey 
(MICS). Tracking progress through these surveys has been 
well characterized for malaria prevention coverage where 
standard indicators have been tracked over time and pro- 
vide a reliable picture of improvements [2]. However, for 
case management, the primary indicator "proportion of 
children under the age of five years with fever in the last 
two weeks who received anti-malarial treatment according 
to national policy within 24 hours from onset of fever" [6] 
has several limitations. These include use of fever as a 
proxy measure for malaria infection, and reliance on care- 
giver reports for information on blood testing and medi- 
cines used for treatment [6]. Further, in the face of recent 
policy changes recommending confirmatory blood testing 
prior to treatment, presumptive treatment of fever is no lon- 
ger a valid indicator for appropriate case management [7]. 
Routine health management information systems (HMIS) 
data capturing malaria blood testing, test results and 
treatments administered are not subject to these limita- 
tions. However, facility data fail to capture coverage 
among the substantial proportion of the population 
where treatment services are not available or among 
those who do not seek care within the public health sys- 
tem. Additionally, in many malaria-endemic countries, 
HMIS are weak, data are incomplete, contain inaccur- 
acies, and are unavailable in an actionable timeframe 
[8]. Finally, HMIS data do not capture characteristics of 
an individual patient's interaction within the health 



system. A facility-based survey designed to assess the ex- 
tent and quality of service provision represents an ap- 
proach that can provide complete and unbiased data in an 
actionable timeframe. Linking facility survey data with 
population-based data could provide a more complete pic- 
ture of malaria case management coverage [8,9]. 

Zambia has made numerous improvements in malaria 
case management in the last decade including adopting 
the ACT artemether-lumefantrine (AL) as the first-line 
treatment for malaria in 2002, and introducing RDTs in 
health facilities in 2006 [10,11]. National guidelines for 
diagnosis and treatment were modified in 2009-10 to 
require parasitological confirmation by RDT or micros- 
copy prior to malaria treatment wherever capacity is 
available [12]. Scale-up of blood testing and AL treat- 
ment was facilitated by substantial donor support. Be- 
tween 2003 and 2010, external partners committed 
nearly $200 million to malaria prevention and control in 
Zambia [10]. Currently, the basic health care package 
offered through the public health system in Zambia 
includes blood testing and AL treatment. These and 
other basic health services are provided free of charge 
in health facilities located in rural and poor districts. 
Zambians access care at three basic levels of public 
health care: rural health posts and community outreach; 
urban and rural health centres; and primary, secondary 
and tertiary hospitals. Nearly all urban households are 
located within 5 km of a health facility as compared with 
50% of rural households. The relatively small private 
for-profit sector operates in urban areas [13] and treat- 
ment seeking in the private sector for suspected malaria 
is relatively uncommon [14]. 

The MIS has been the primary tool for tracking recent 
malaria control progress in Zambia; four rounds have 
been implemented in 2006, 2008, 2010, and 2012. In 
2011, a nationally representative Health Facility Survey 
(HFS) was implemented. 2011 HFS and 2012 MIS data 
and a systems effectiveness approach were used to docu- 
ment malaria case management coverage in Zambia 
following recent improvements. A systems effectiveness 
framework evaluates coverage by examining the neces- 
sary sequential components for effective health service 
delivery [15]. This framework documents the extent to 
which highly efficacious disease control tools effectively 
reduce disease burden. Effectiveness is a function of 
whether a disease control tool reaches the target group; 
users and providers comply and perform correctly; and a 
high level of coverage is sustained [16]. Effectiveness of 
highly efficacious anti-malarial drug (i.e. ACT) is a func- 
tion of: 1) treatment-seeking behaviour among febrile 
individuals - seeking treatment from an appropriate pro- 
vider; 2) malaria blood testing provided to patients with 
fever to confirm malaria infection; 3) appropriate treat- 
ment with ACT based on blood test results (provider 
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compliance); and 4) patient adherence to anti-malarial 
treatment [17-20]. Consequently, the information required 
to use this approach must come from multiple sources, 
both from population data (using representative house- 
hold surveys) and facility data (using representative health 
facility surveys). Benchmarking systems effectiveness can 
guide Zambia stakeholders towards action to improve 
malaria case management. 

Methods 

2012 Malaria Indicator Survey (MIS) 

Details on the Zambia MIS methods are described else- 
where [21]. Briefly, the MIS is a national population-based 
household survey designed to measure key indicators for 
tracking malaria control progress. A random sample of 
households stratified by urban and rural residence was 
drawn using two-stage cluster sampling. Standard enumer- 
ation areas were selected with probability-proportional-to- 
size at the first stage and a random sample of 25 households 
per cluster was selected at the second stage. 

Within each sampled household, a standard question- 
naire administered using a personal digital assistant 
(PDA), collected basic information about the household 
and each household member. The MIS Household and 
Woman's Questionnaires are modeled after the survey 
instruments developed by the MEASURE DHS + pro- 
gramme and adopted and recommended for use by the 
Roll Back Malaria Monitoring and Evaluation Reference 
Group Task Force on Household Surveys [22]. The stand- 
ard household roster was expanded to include questions 
about recent febrile illness, fever treatment-seeking behav- 
iour, and anti-malarial treatment among all household 
members. Each woman between the ages of 15 and 49 
completed a standard woman's questionnaire interview 
that included questions about each of her children under 
age five. These questions included information about 
febrile illness occurring in the two weeks preceding the 
survey, treatment-seeking behaviour, blood testing, and 
anti-malarial treatment received for the fever. Where par- 
ental consent was obtained, children were administered an 
RDT and specimens were collected on a blood slide (BS) 
for microscopy [22]. 

Fieldwork was completed between April and May, 
2012; 3,800 households participated in the survey. 
Within these households, information was collected in 
the household questionnaire on 16,928 household mem- 
bers, including 2,820 household members with recent 
fever. Information about 2,620 children under five was 
collected in the woman's questionnaire from 2,301 bio- 
logical mothers age 15-49, including 742 children with 
recent fever. 

All MIS and HFS research activities were reviewed by 
the Research Ethics Committee of the University of 
Zambia on behalf of the Ministry of Health, the PATH 



Research Ethics Committee, and the US Centers for 
Disease Control Institutional Review Board. Final au- 
thority to conduct these research activities was obtained 
from the Ministry of Health. 

2011 Health Facility Survey (HFS) 

Details on the Zambia HFS methods are described else- 
where [23]. Briefly, the HFS was designed to provide 
nationally representative information on facility-based mal- 
aria case management. The study was modeled after the 
World Health Organization Integrated Child Health Facility 
Assessment [24] and applications in similar contexts [25]. 

The sampling frame consisted of a list of all health 
facilities in Zambia compiled in 2009 by the Ministry of 
Health (N = 1,843). Most health facilities in Zambia are 
government-run (85%), although a few are run by private 
for-profit (9%) or mission organizations (6%) [13]. As 
such, the sample represents health facilities in Zambia, 
with chance of including private and mission facilities 
equivalent to their representation among all national 
health facilities. The sample was drawn with explicit strati- 
fication from lists of all facilities at each level of care: 
health post (N = 266), urban health centre (N = 428), rural 
health centre (N = 1,042), and hospital (N = 107). Within 
each group, geographic stratification was done by ordering 
facilities by location (province, district), and a random 
sample of health facilities was selected using systematic 
random sampling. Forty-two facilities were sampled from 
each stratum for a total of 168 facilities. The overall facility 
response rate was 89% (N = 149). The selected sample was 
largely government (75%) and non-profit/mission facilities 
(11%); 12% of the sampled facilities were private for-profit 
and 3% were facilities for company employees (i.e., com- 
mercial farm or mining companies). 

Each facility was surveyed for one day during which a 
sample of patients was selected. Inclusion criteria for the 
study were as follows: patients visiting the outpatient 
clinic for consultation with a health worker and present- 
ing at the health facility for the current illness for the 
first time. There were 1,464 patients identified as eligible 
for the study. A participation rate of 95% (N = 1,394) 
was achieved. Of these eligible patients, consultation 
observation and exit re-assessment were performed for 
1,290 patients who were reportedly seeking care for the ill- 
ness for the first time. Of these patients, 872 had suspected 
malaria (self-reported fever or a history of fever and/or 
body temperature greater than or equal to 37°C), including 
patients seeking care at hospitals (N = 213), urban health 
centres (N = 221), rural health centres (N = 251), and 
health posts (N = 187). One-half (51%) of the patients with 
suspected malaria were under age five years. 

Two patient-focused survey instruments were used: 1) 
consultation observation form to capture case manage- 
ment practices; and, 2) patient exit interview and re- 
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examination. The silent consultation observation was 
completed first; laboratory tests and results for the pa- 
tient were recorded on this form. After the patient fully 
completed the consultation, the exit interview was con- 
ducted to assess patient comprehension of the diagnosis 
and prescribed drug regimens as well as patient recall 
and assessment of health worker performance during 
the consultation. Finally, a patient re-examination per- 
formed by a trained health professional provided a gold 
standard malaria diagnosis (clinical assessment con- 
firmed by RDT). 

Measurement and analysis 

HFS indicators were created among the subsample of 
patients with suspected malaria. Suspected malaria was 
determined at an exit re-assessment and was defined as 
presence of self-reported fever or a history of fever and/ 
or body temperature greater than or equal to 37°C. A 
variable was created to indicate whether or not these pa- 
tients received a blood test during consultation accord- 
ing to the consultation observation form (interviewer 
observation and/or information recorded from patient 
records). Treatment with the national first-line ACT 
(AL) was measured at exit interview by review of patient 
medicines and/or prescriptions obtained during the facil- 
ity visit. Variables concerning patient comprehension of 
the drug regimen were created among those patients 
who obtained AL at the health facility using questions 
regarding the duration of treatment (number of days), 
times per day the medicine was to be given, and dosage 
(number of tablets). The respondent's answers were con- 
sidered with respect to the correct dosage determined 
using patient age and weight information. 

MIS indicators focus on treatment-seeking behaviour 
and outcomes among people with fever in the two weeks 
preceding the survey. This sample of recently febrile in- 
dividuals included children under five years of age using 
reports from women age 15 to 49 about their own bio- 
logical children as well as all household members from 
the household listing. While the 2012 MIS survey report 
focuses on fever treatment-seeking behaviour for chil- 
dren under five [21], this paper uses secondary data 
analysis to produce estimates for people of all ages. Indi- 
cators were calculated from the household listing for all 
household members, except in the case of children 
under five with information available from the mother's 
report, which serves as the standard source of fever 
prevalence and treatment-seeking behaviour used by 
major household survey groups. Variables were created 
to indicate treatment seeking for recent fever at an 'ap- 
propriate provider' - defined as any public or private 
health facility or a community health worker (CHW). 
Treatment-seeking and ACT treatment were calculated 
among all febrile individuals, however information on 



blood testing (recall of a finger or heel stick as a stand- 
ard proxy measure for a malaria blood test) was only 
available for children under five recorded in the woman's 
questionnaire. 

Tabulations and cross tabulations were performed using 
Stata 12.1 (© StataCorp, College Station, TX, USA). Point 
estimates and 95% confidence intervals (CI) are reported. 
Data analyses took into account the survey design: MIS 
stratification (urban/rural) and clustering of households 
within standard enumeration areas, and HFS stratification 
(level of care) and clustering of patients within health facil- 
ities. All data were weighted to account for disproportion- 
ate allocation of the sample to different strata. 

An overall measure of systems effectiveness was calcu- 
lated across key indicators on the pathway from ACT ef- 
ficacy to effectiveness as follows: systems effectiveness = 
[AL efficacy (documented 100% [26])] x [% people with 
current or recent fever who reportedly sought treatment 
from an appropriate provider (MIS)] x [% patients with 
current or recent febrile illness seeking care at an appro- 
priate provider who received a malaria blood test (HFS)] 
x [% patients with a positive blood test who received AL 
(HFS)] x [% patients prescribed AL who demonstrated 
full comprehension of the treatment regimen (HFS, 
proxy for adherence)] [16]. 

Results 

2012 Malaria Indicator Survey (MIS) 

The MIS captures self-reported fever management out- 
comes among people with fever in the two weeks prior 
to the household survey. More than half of children 
under five years of age (63%) and people age five years 
and above (58%) sought fever treatment outside of the 
home. Treatment from an appropriate provider (CHW, 
public health facility, or private health facility) was 
sought among more than half (55%) of children under 
five years and 44% of people age five years and above 
(Figure 1). Among people seeking treatment from an ap- 
propriate provider (N = 1,257), 71% reportedly visited a 
public health centre; 15% visited a public hospital; 4% 
visited a public health post; 1% visited a CHW; and 7% 
visited a private facility. Seeking fever treatment from 
other private outlets is reportedly uncommon in Zambia 
(10% of children under five and 13% of people age five 
and above) (Figure 1). Among people seeking treatment 
in other private outlets (N = 333), 71% reportedly visited 
a shop and 28% visited a pharmacy. 

The MIS measured blood testing among children 
under five recorded in the woman's questionnaire. 
Among children with recent fever, 31% reportedly re- 
ceived a blood test for malaria. Blood testing varied ac- 
cording to where children were taken for treatment. 
Among children managed exclusively by an appropriate 
provider, 61% received a blood test. Among children 
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Age 0-4 Age 5+ 

(N=972) (N=1848) 

■ Any source outside of the home 

□ Appropriate provider (CHW, public or private facility) 

■ Public facility 

■ CHW 
Private facility 

Other private (pharmacy, shop) 

Figure 1 Percentage of people with recent fever who reportedly sought treatment outside of the home, across age. Source: MIS 2012 
household roster and woman's questionnaire. 



managed exclusively by a pharmacy or shop, only 2% re- 
portedly had a blood test (Figure 2). 

Figure 3 examines use of ACT relative to other anti- 
malarials reportedly used for recent fever. Among those 
reportedly treated with any anti-malarial, 86% of chil- 
dren under five and 78% of people age five and above 
received an ACT. ACT treatment was higher among 
people exclusively managed by an appropriate provider 
(86% of children under five, 81% of people age five and 
above) as compared with people managed by pharmacies 
or shops (72% of children under five, 46% of people age 
five and above) (Figure 3). 

Malaria RDTs were administered and BS specimens for 
microcopy were collected from children under five who 
were recorded in the woman's questionnaire. Among re- 
cently febrile children who did not seek treatment from an 
appropriate provider, current infection was identified in 
the 30% who were both BS and RDT positive. These re- 
sults indicate that a substantial fraction of children with 
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»> ™ S3 
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% People with fever 



Figure 2 Percentage of children under five with recent fever 
who reportedly received a blood test, across provider type. 

Source: MIS 2012 woman's questionnaire. 1 CHW, public or private 
facility. 2 Pharmacy, shop. 



febrile illness who are not taken to an appropriate provider 
actually harbour malaria infection - an infection that may 
persist untreated, contributing to transmission, ongoing 
morbidity, and potentially severe illness and even death. 
About half of recently febrile children who did not seek 
treatment from an appropriate provider had no evidence 
of malaria infection (44% BS and RDT negative), and an 
additional 25% were BS negative and RDT positive - pos- 
sibly indicating recent anti-malarial treatment with medi- 
cations that were stored at home or acquired from 
another source (Table 1). 

Among children who were taken to an appropriate 
provider for fever treatment, 16% remained BS and RDT 
positive at the time of the survey, and an additional 33% 
were BS negative and RDT positive (Table 1). The frac- 
tion of children who reportedly received care from an 
appropriate provider but nonetheless remained BS posi- 
tive at the time of the survey suggests that these infec- 
tions were not effectively managed. 

Among children under five years who were BS and 
RDT positive, fewer than half (44%) reportedly expe- 
rienced fever in the two weeks preceding the survey. 
Prevalence of recent fever among infected children 
declines with age from 61% to 37% between ages 12 to 
23 months to age 48 to 59 months (Table 2). 

2011 Health Facility Survey (HFS) 

The HFS documents observed fever management out- 
comes among patients with suspected malaria (fever or 
history of fever) seeking treatment at a health facility. 
Among patients with suspected malaria exiting public 
health facilities in 2011, 71% of children under five years 
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Figure 3 Percentage of people with recent fever treated with an anti-malarial who received ACT treatment, across age and provider 
type. Source: MIS 2012 household roster and woman's questionnaire. 1 CHW, public or private facility. 2 Pharmacy, shop. 



and 64% of people age five years and above received a 
blood test for malaria. Data trends show higher testing 
rates among patients managed in hospitals and urban 
health centres as compared with rural health centres and 
health posts (Figure 4). 



Among patients tested for malaria during the consultation, 
230 (39%) tested positive and 378 (61%) tested negative. 
Among patients who tested positive during consultation, 
86% of children under five and 89% of people age five and 
above were prescribed ACT. Low levels of ACT prescription 



Table 1 Malaria infection among recently febrile children, across treatment-seeking behaviour and age 



Did not seek treatment from an appropriate provider 1 




All children 


0-11 months 


12-23 months 


24-35 months 


36-47 months 


48-59 months 


N fever 


443 


62 


95 


85 


113 


88 


% BS-, RDT- 


44.1 


70.3 


44.3 


53.4 


32.2 


32.2 




(36.1-52.5) 


(57.8-80.3) 


(31.2-58.2) 


(38.5-67.8) 


(22.1-44.2) 


(20.3-46.9) 


% BS+, RDT+ 


29.9 


17.4 


39.0 


27.3 


31.0 


31.0 




(24.1-36.6) 


(9.7-29.3) 


(26.3-53.3) 


(1 7.6-39.8) 


(22.3-41.4) 


(22.5-41.0) 


% BS-, RDT+ 


24.8 


9.4 


16.8 


19.3 


34.7 


35.7 




(19.4-31.0) 


(3.4-23.4) 


(1 1 .0-24.8) 


(11.5-30.5) 


(25.8-44.7) 


(23.7-49.8) 


% BS+, RDT- 


1.2 


2.9 


0.0 


0.0 


2.1 


1.2 




(0.5-3.0) 


(0.7-11.1) 






(0.7-6.6) 


(0.2-8.3) 


Sought treatment from an appropriate provider 1 




All children 


0-11 months 


12-23 months 


24-35 months 


36-47 months 


48-59 months 


N fever 


--■151 


74 


86 


110 


103 


78 


% BS-, RDT- 


49.8 


68.8 


59.2 


45.3 


41.6 


34.1 




(41.9-57.7) 


(56.2-79.1) 


(43.9-72.9) 


(34.1-57.0) 


(30.1-54.0) 


(22.5-48.0) 


% BS+, RDT+ 


16.2 


6.1 


13.1 


21.9 


16.4 


23.4 




(12.6-20.7) 


(2.9-12.3) 


(7.7-21.4) 


(14.4-32.0) 


(10.7-24.4) 


(14.3-35.9) 


% BS-, RDT+ 


33.2 


23.1 


27.7 


32.8 


41.2 


41.3 




(27.1-40.0) 


(14.9-34.0) 


(17.9-40.3) 


(23.5-43.7) 


(29.3-24.4) 


(30.5-53.0) 


% BS+, RDT- 


0.7 


2.1 


0.0 


0.0 


0.8 


1.2 




(0.3-2.0) 


(0.5-8.1) 






(0.1-5.8) 


(0.2-8.0) 



Source: MIS 2012 household roster and woman's questionnaire. 
1 CHW, public or private facility. 
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Table 2 Recent fever among infected children, across age 




N BS and RDT+ 


% Recent fever (95% CI) 


All children 


512 


43.7 (36.8-50.9) 


Age 






0-1 1 months 


46 


39.0 (25.2-54.8) 


12-23 months 


85 


61.2 (47.6-73.2) 


24-35 months 


122 


43.4 (33.9-53.3) 


36-47 months 


124 


42.4 (31.7-53.8) 


48-59 months 


135 


36.5 (26.5-47.8) 



Source: MIS 2012 household roster and woman's questionnaire. 



were observed among patients that tested negative during 
consultation: 11% of children under five and 12% of people 
age five and above (Figure 5). Use of other anti-malarials 
(quinine, sulphadoxine-pyrimethamine (SP)) to treat sus- 
pected malaria cases was infrequently observed among all 
patients including those who were not tested during consult- 
ation (3%); patients with a negative blood test result (3%); 
and patients with a positive blood test result (3%). 

Patients exiting with a prescription for AL were asked to 
explain the drug regimen, including dosage (number of 
tablets based on age/weight), doses per day (two times), 
and number of days to give the drug (three days). Complete 
correct knowledge of the regimen was demonstrated by 
73% of caregivers of children under five and 61% of care- 
givers/patients age five and above (Figure 6). 

Systems effectiveness for malaria case management 

The overall systems effectiveness for malaria case man- 
agement is 25% for children under five years and 15% 
for people age five years and above (Figure 7). These re- 
sults are influenced most heavily by losses at the first 
step in the case management cascade - low rates of 



seeking treatment from an appropriate provider. More 
than half (56%) of people age five years and above and 
45% of children under five years of age with suspected 
malaria (i.e. current or recent fever) are lost at this stage. 
Blood test results among children under five indicate 
that the loss is considerably smaller (14%) when consid- 
ering only those children who had positive RDT and BS 
results (Figure 7). 

Discussion 

Documenting malaria case management progress in the 
context of recent ACT and RDT scale-up has been a 
challenge given limitations of the most commonly avail- 
able data sources - population-based household surveys 
(MIS, DHS) and HMIS data. The challenge to document 
progress in Zambia was addressed by applying a systems 
effectiveness approach using the available national 
household survey data (2012 MIS), and the recently 
available national health facility survey data (2011 HFS). 
Assembling results from these surveys, overall systems 
effectiveness was estimated at 25% for children under 
five, and 15% for people age five and above. In other 
words, malaria diagnosis and treatment services in 
Zambia are effective in managing 25% and 15% of sus- 
pected cases under age five and age five years and above, 
respectively. Tracking the steps from efficacy to systems 
effectiveness highlighted areas that require strengthening 
to improve case management coverage. 

The first step in the cascade from ACT efficacy to 
systems effectiveness is seeking treatment from an ap- 
propriate provider. In this context, appropriate providers 
include public or private health facilities and CHWs. 
These sources are the focus of national policies, guide- 
lines, and support for malaria case management. The 
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Source: HFS 201 1. 1 Self-reported fever or a history of fever and/or body temperature greater than or equal to 37°C 



definition of an appropriate provider should be modified 
according to the set of promoted providers in a given 
country. Failure to seek treatment from an appropriate 
provider represents the largest threat to systems effect- 
iveness in Zambia; only about half of people with recent 
fever reportedly sought care from an appropriate pro- 
vider. This gap suggests a need for more information on 
barriers and facilitating factors for fever treatment- 
seeking behaviour in this context. The MIS is well suited 
to estimating levels of and at least some of the factors 
determining treatment-seeking behaviour for recent 



fever. These include a measure of household socio- 
economic status, child characteristics including age, and 
mother's education, malaria knowledge, and exposure to 
malaria communications, and household distance to a 
health facility or appropriate care provider. This infor- 
mation can be used to improve intervention targeting. 

To address gaps in treatment-seeking behaviour, MIS 
and other population-based data on treatment-seeking 
behaviour should be reviewed in the context of national 
programme information on service availability. For ex- 
ample, partners in Zambia have focused resources on 
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improving access to malaria case management services 
through extension of diagnosis and treatment to the 
community level. CHWs are recruited, trained and su- 
pervised to provide malaria-specific or integrated pack- 
ages of community-based case management [27]. Results 
from the 2012 MIS suggest that, at national level, fever 
cases are not typically managed by these community 
agents (only 2% of children under five with recent fever). 
However, given that community case management pro- 
gramme coverage is subnational, additional analyses 
should evaluate CHW service utilization explicitly in 
areas with current programme coverage. These analyses 
would provide information on the extent to which the 
strategy facilitates appropriate treatment-seeking behav- 
iour. Assembling existing data on service availability and 
utilization can facilitate targeted investment in scale-up to 
improve national coverage. Where existing data cannot 
adequately explain underutilization of accessible care 
options, additional information about treatment-seeking 
behaviour may be needed. Qualitative research, often 
entailing use of rapid ethnographic methods, is commonly 
employed to understand the complexities and variations 



in illness identification and treatment-seeking behaviour. 
Recent reviews suggest that such studies could be 
strengthened through application of theory and longer- 
term engagement through methods such as extended case 
studies and ethnography [28-30]. 

The second systems effectiveness step is blood testing 
for suspected malaria. Frequency of blood testing as 
measured in the Zambia HFS was high; 71% of patients 
under age five and 64% of patients age five years and 
above received a blood test. The MIS also provides a 
rough approximation of blood testing in this context 
and suggests that 61% of children under five taken to an 
appropriate provider were tested for malaria. Findings 
from a recent Zambia study to validate standard MIS 
questions also suggest that the MIS may underestimate 
blood testing [9]. While both surveys show impressive 
improvement in malaria testing coverage in Zambia 
[31,32], gaps persist. Health facility assessments provide 
an opportunity for measuring a number of factors that 
may influence blood-testing practices; these include 
facility factors (e g, availability of RDTs, microscopy, na- 
tional guidelines, and job aids, level of care, staffing), 
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provider factors (e g, training, supervision, experience, sat- 
isfaction, client case load), and patient factors (e g, age, 
other symptoms). 

The third essential step for systems effectiveness is 
ACT treatment for people with confirmed malaria infec- 
tion. Zambia HFS results indicate tremendous progress 
in this area - most people who test positive for malaria 
are prescribed ACT (86% for children under five and 
89% for age five and above). The HFS is unique in its 
ability to determine rates of ACT prescription based on 
test results. Additionally, HFS results in Zambia indicate 
that in most cases, ACT is dispensed rationally based on 
test results; ACT was prescribed to just one in ten pa- 
tients who tested negative for malaria. Tracking rational 
use can help explain reductions in systems effectiveness 
at this stage where over prescription may contribute not 
only to inefficiencies, but to stock-outs that ultimately 
prevent ACT prescription for positive cases. ACT use 
based on test results is not well captured in household 
surveys. The MIS can include questions about malaria 
blood test results, however patients seeking care may 
not be adequately and accurately informed about test re- 
sults and/or may not accurately recall the result [9]. 
Routine collection of facility data through the HMIS 
could capture information on testing and treatment. 
However, in the Zambian context, the HMIS collects 
aggregate reports on numbers of patients tested and treat- 
ments dispensed, rather than individual patient data on 
testing, diagnosis, and treatment. Efforts to strengthen the 
reporting and linkages of data on stock management and 
disease reporting could facilitate additional triangulation 
of information for review on a routine basis. 

With the wide recognition that the previous global in- 
dicator based on presumptive anti-malarial treatment of 
fever is no longer informative, a new indicator proposed 
by the Roll Back Malaria Monitoring and Evaluation 
Reference Group is "the proportion of people receiving 
an ACT, among people with fever in the last two weeks 
who received any anti-malarial drugs [7]". Applying this 
indicator in Zambia, most anti-malarial treatments re- 
portedly received are ACT (86% for children under five 
and 78% for age five and above), particularly where treat- 
ment is acquired from an appropriate provider. In other 
contexts, where the anti-malarial market is diverse both 
in terms of market share and in types of anti-malarials 
in circulation, this MIS indicator could provide informa- 
tion on anti-malarial prescription among various pro- 
vider types to supplement a more targeted facility survey 
focused on tracking behaviour among certain providers. 
Nonetheless, the MIS ACT indicator denominator is only 
among those individuals who received any anti-malarial 
and does not permit evaluation of those not receiving an 
anti-malarial for whatever reason. The HFS can track ap- 
propriate and rational ACT use, and can capture and 



explore facility, provider, and patient factors associated 
with prescription behaviour to inform intervention. 

The final step in the systems effectiveness framework 
is patient adherence to the drug regimen. Data on drug 
adherence are often imperfect; the behaviour is difficult 
to measure, self- report measures may be unreliable [33], 
and a 2005 review noted that there is insufficient infor- 
mation on adherence to anti-malarials [34]. In the ab- 
sence of a direct measure of adherence to AL in Zambia, 
a proxy measure was used - comprehension of the drug 
regimen at exit from a health facility. While the relation- 
ship between comprehension and adherence may not be 
exact, a recent study of AL adherence in Kenya iden- 
tified correct knowledge of the drug regimen as the 
strongest predictor of adherence [35]. To fully estimate 
effective case management intervention coverage, and to 
monitor behaviours that can contribute to anti-malarial 
drug resistance, additional study designs and instru- 
ments are needed. 

The systems effectiveness approach to assemble 
household and facility data appears to provide the most 
complete picture of malaria case management to date. 
However, one of the limitations in linking these data 
sources through this approach is in treating self- 
reported fever as a proxy and denominator for sus- 
pected malaria in the overall calculation of systems 
effectiveness. The approach assumes that febrile indi- 
viduals with malaria infection are no more or less likely 
to seek treatment than febrile individuals who do not 
harbour malaria infection. Following this logic, all fevers 
must be managed by an appropriate provider in order to 
reach all malaria cases. In reality, the malaria-attributable 
fraction of febrile cases varies across seasons and contexts, 
and declines with scale-up of control interventions [36]. 
In treating all reported fevers the same, the approach 
underestimates the overall systems effectiveness for 
managing true symptomatic parasitaemic cases. Indica- 
tion of this under-estimation is apparent in examining 
blood test results for children with fever who were not 
taken to an appropriate provider; nearly half were both 
BS and RDT negative. It is also important to note that 
this framework is limited in its application to manage- 
ment of symptomatic malaria infections. Focusing on 
symptomatic cases to reduce severe disease and death is 
appropriate in the context of malaria control in high 
transmission settings. However, asymptomatic (afebrile) 
infections contribute to ongoing transmission as well as 
disease burden [37,38]. Analysis of MIS blood test re- 
sults among children under five indicates that fewer 
than half (44%) of children that were both BS and RDT 
positive reportedly experienced recent fever. Clearing 
malaria infections - both symptomatic and asymptom- 
atic - will require different intervention strategies and 
different metrics to document progress [39-41]. 
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This study used standard household and facility survey 
instruments and methods, and data were collected among 
a nationally representative sample. Despite these strengths, 
each data source has limitations. First, while each study 
provided reasonably precise estimates for key indicators, 
comparisons across age groups and levels of care resulted 
in loss of precision (i e, wide confidence intervals) due to 
smaller sample sizes among these subgroups. Sample size 
calculations for facility surveys are challenging given a 
number of factors that influence the ultimate numbers of 
study participants among each subgroup of interest, in- 
cluding cases of suspected malaria, confirmed cases, and 
treated cases. Measurement bias is another limitation in 
both studies. Household surveys estimate aspects of cover- 
age using respondent reports, which are subject to recall 
bias. This bias is minimized in the MIS and other standard 
surveys by administering questions regarding fever that 
occurred recently - in the two weeks prior to the survey. 
Initial studies aimed at validating respondent recall and re- 
sponse suggest need for further research [9]. Consultation 
observation and patient exit interviews were used in the 
HFS. These methods are subject to the Hawthorn effect - 
a bias that may arise due to study participant awareness of 
being under observation; health workers may perform bet- 
ter than usual given presence of study teams [42,43] . This 
study facilitated triangulation of HFS results with MIS 
findings to some extent (e g, regarding blood testing for 
suspected malaria), thereby strengthening conclusions. 
Neither data source provided direct measure of adherence 
to AL - a major limitation to completing the systems 
effectiveness framework. Further research is needed on 
this behaviour in Zambia. Data to estimate systems effect- 
iveness were collected through two discrete surveys, one 
in 2011 (HFS) and one in 2012 (MIS). Temporal dif- 
ferences are important to consider when assembling data 
from different surveys because in many contexts, case 
management interventions and coverage are rapidly evolv- 
ing. While data from Zambia used in this analysis were 
not collected simultaneously, the systems they were evalu- 
ating were relatively stable over the time period encom- 
passing both surveys, facilitating meaningful combination 
of survey results. While the systems effectiveness frame- 
work highlighted the extent to which gaps exist in the 
pathway from efficacy to effectiveness, there remains a 
need to delve deeper into each step to inform interven- 
tions. Specific strategies for improving treatment-seeking 
behaviour, as well as blood testing, rational AL use, and 
patient adherence, can be developed based on further 
studies of factors associated with these outcomes. 

Applications of the systems effectiveness framework to 
benchmark progress in other country contexts may 
require additional data beyond what was applied in this 
study. Investments to improve malaria case management 
in Zambia have focused primarily on public health 



facilities and CHWs, augmented by a relatively small 
contingent of private health facilities. Among this set of 
appropriate providers, 2012 MIS data indicate that 
health facilities were commonly accessed while CHWs 
were an infrequent source of care for fever management 
at national level. Thus fever management components of 
the systems effectiveness framework were completed 
using information from health facilities. In comparison, 
health systems for case management in other malaria 
endemic countries are more diverse; appropriate pro- 
viders who are authorized, trained, and promoted to 
manage malaria may include private sector providers in- 
cluding pharmacies, drug shops, and mobile vendors 
[44,45]. Although the health system for malaria case 
management is unique in each country, benchmarking 
progress using a systems effectiveness framework seems 
applicable across contexts. Where the system is com- 
prised of multiple actors operating in community, pub- 
lic, and private spheres, assembling a picture of systems 
effectiveness will require information first on the extent 
to which each component of the system is accessed for 
care; and second, information on case management 
(blood testing and drug prescription) within each of the 
various system components. Ultimately, the utility of the 
systems effectiveness framework lies in the simplicity of 
assembling and interpreting data from multiple sources. 
This utility should be tested in the context of diverse 
health systems for malaria case management. 

Conclusion 

Health facility and population-based household survey 
data (HFS and MIS) applied to the systems effectiveness 
framework provides a cogent and feasible approach to 
documenting malaria case management coverage. In 
Zambia, these results indicate that the largest threat to 
systems effectiveness is appropriate treatment-seeking 
behaviour - which can encompass both access to ap- 
propriate case management service providers as well as 
service utilization. Although room for improvement re- 
mains, study results demonstrate high facility coverage 
of blood testing for patients with symptoms of malaria 
and high rates of rational ACT use. Further analysis of 
the MIS and HFS can identify specific factors associated 
with treatment-seeking behaviour, blood testing, and ra- 
tional AL use and can inform interventions to improve 
malaria diagnosis and treatment. Until now, tracking 
progress in malaria case management coverage has re- 
lied heavily on population-based surveys. The way for- 
ward likely entails household surveys to track trends in 
treatment-seeking behaviour, and facility/provider data 
(survey, routine, or supervisory data) to track appropri- 
ate management of suspected malaria cases. Finally, as 
programmes embark on the road to malaria elimination, 
blood-testing components of household surveys can 
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document the extent to which infections are symptom- 
atic (febrile) and recent fever is an indicator of infection. 
This information can shape appropriate interventions to 
target both symptomatic and asymptomatic infections. 
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